Graph-embedding Enhanced Attention Adversarial Autoencoder

When dealing with the graph data in real problems, only part of the nodes in the graph are labeled and the rest are not. A core problem is how to use this information to extend the labeling so that all nodes are assigned a label (or labels). Intuitively we can learn the patterns (or extract some representations) from those labeled nodes and then apply the patterns to determine the membership for those unknown nodes. A majority of previous related studies focus on extracting the local information representations and may suffer from lack of additional constraints which are necessary for improving the robustness of representation. In this work, we presented Graph- embedding enhanced attention Adversarial Autoencoder Networks (Great AAN), a new scalable generalized framework for graph-structured data representation learning and node classification. In our framework, we firstly introduce the attention layers and provide insights on the self-attention mechanism with multi-heads. Moreover, the shortest path length between nodes is incorporated into the self-attention mechanism to enhance the embedding of the node’s structural spatial information. Then a generative adversarial autoencoder is proposed to encode both global and local information and enhance the robustness of the embedded data distribution. Due to the scalability of our approach, it has efficient and various applications, including node classification, a recommendation system, and graph link prediction. We applied this Great AAN on multiple datasets (including PPI, Cora, Citeseer, Pubmed and Alipay) from social science and biomedical science. The experimental results demonstrated that our new framework significantly outperforms several popular methods

Network Sketching: Exploiting Binary Structure in Deep CNNs

Convolutional neural networks (CNNs) with deep architectures have substantially advanced the state-of-the-art in computer vision tasks. However, deep networks are typically resource-intensive and thus difficult to be deployed on mobile devices. Recently, CNNs with binary weights have shown compelling efficiency to the community, whereas the accuracy of such models is usually unsatisfactory in practice. In this paper, we introduce network sketching as a novel technique of pursuing binary-weight CNNs, targeting at more faithful inference and better trade-off for practical applications. Our basic idea is to exploit binary structure directly in pre-trained filter banks and produce binary-weight models via tensor expansion. The whole process can be treated as a coarse-to-fine model approximation, akin to the pencil drawing steps of outlining and shading. To further speedup the generated models, namely the sketches, we also propose an associative implementation of binary tensor convolutions. Experimental results demonstrate that a proper sketch of AlexNet (or ResNet) outperforms the existing binary-weight models by large margins on the ImageNet large scale classification task, while the committed memory for network parameters only exceeds a little.Comment: To appear in CVPR201